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Overview 


►  Purpose 

►  PSR  Process 

-  Purpose 

-  PSR  policy 

-  Notional  PSR 

-  Example  Finding 

►  Acquisition  ESOH  Observations 

►  Value 

►  Path  Forward 
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Purpose 

►  This  briefing  provides  an  overview  of  the  current  efforts  by  the 
Office  of  the  Deputy  Under  Secretary  of  Defense  (Installations 
&  Environment)  through  the  DoD  Acquisition  Environment, 
Safety,  and  Occupational  Health  (ESOH)  Integrated  Product 
Team  (IPT)  to: 

>  Participate  in  Program  Support  Reviews  (PSRs) 

-  Gauge  policy  compliance 

-  Assess  policy  effectiveness 

-  Provide  Immediate  guidance  (improvements)  to  Programs, 
as  needed 
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Purpose  -  Support  Defense  Acquisition  Management  System 
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Program  Support  Reviews 

►  DASD(SE)  leads  Program  Support  Reviews  (PSRs) 

-  Systems  Engineering  look  at  Programs  before  a  Milestone  decision 

-  Assessment/review  of  Program  against  OSD  Policy 

-  Examines  multiple  aspects  of  a  Program 

►  ODUSD(l&E)  is  providing  ESOH  Subject  Matter  Experts  and 
coordinating  with  DASD(SE) 

►  Utilizing  body  of  knowledge  from  DoD  Acquisition  ESOH  IPT 

-  ODUSD(l&E)  leads  ESOH  SME  team 

-  Services  provide  Acquisition  ESOH  Principal’s  support  to  PSRs  for 
which  their  service  is  the  lead 
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ESOH  in  PSRs  Guidance  Documents 

►  DoDI  5000.02  Operation  of  the  Defense  Acquisition  System 

-  Enclosure  2  -  Procedures 

-  Enclosure  12  -  System  Engineering 

►  Defense  Acquisition  Guidebook  (DAG) 

►  Defense  Acquisition  Program  Support  (DAPS)  Methodology 
(Guide) 

-  Section  4.0,  Technical  Processes 

Sub-Area  4.1,  Design  Considerations 
Factor  4. 1.4,  ESOH 

Factor  4.1 .7,  Corrosion  (Hexavalent  Chromium) 
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Review  Areas  of  PSRs  -  DAPS  Methodology 


1 .  Mission  Capabilities  -  Clarity  and  stability  of  CONOPS,  mission  requirements,  and  implication  for 
system  requirements  /  constraints,  program  structure  and  execution. 

2.  Resources  -  Budget  sufficiency  and  phasing,  staffing,  system  schedule,  and  assets  available  to 
meet  program  objectives. 

3.  Management  -  Acquisition  strategy  and  planning,  criteria,  contracting,  risk,  tools,  and  techniques 
used  to  manage  the  program. 

4.  Technical  Processes  -  Design  considerations,  requirements  development,  technical  baselines, 
engineering  tools,  software,  design  verification,  and  producibility  and  supportability  planning  for 
product  development. 

5.  Performance  -  Effectiveness  and  Suitability  maturity  and  adequacy  of  product(s)  and  services 
being  acquired  (includes  hardware,  software,  production  considerations  and  logistics  support). 

6.  Special  Interest  Areas  -  Request  For  Proposal,  etc. 
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Taxonomy  of  Classifications 


+  Positive 
O  Neutral 
-  Negative 
■“  Issue 
-  Risk 


Note:  When  recording  multiple  negatives  in  a  PSR  report,  ensure  that  each  negative 
has  clear  linkage  with  its  risk  or  issue,  recommendation,  root  cause,  and  impact 
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Program  Support  Review  (Stoplight  Summary) 
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DAPS-level  results 

13  positive  findings 
16  neutral  findings 
30  negative  findings 
57  issues 
35  risks 

46  recommendations 
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Top-Level  Program  Risks  (PSR  team) 


Risk:  Transition  Planning 

Drivers: 

•  Transition  Support  Plan  lacks  details  for  adoption  of 
MS  processes  and  procedures 

•  Potential  Concept  of  Employment  (CONEMP) 
differences  (C,  S) 

Recommendations: 

□  MS  get  PCO  on-board,  conduct  detailed  review  of 
contract,  identify  /  implement  changes 

□  Program  identify  process  differences  and  planning 
gaps  in  Transition  Support  Plan 


Risk:  Sustainment  Planning 

Drivers: 

•  Inadequate  sustainment  planning  at  program 
inception,  RMD  802  forces  re-evaluation  (C,S) 
o  BCA  late-to-need  for  supportability  decision 
o  No  visibility  into  repairs  and  FRACAS  for 

components  below  line-replaceable-unit  level 

•  Insufficient  plan  for  design  sustainment  (C,P) 
o  Lack  of  defined  block-upgrade  strategy 

o  ESOH,  PESHE  and  Corrosion  plans  are  incomplete 
Recommendations: 

□  Program  update  technical  documentation:  SEP,  AS, 
MOSA,  PESHE,  etc. 

□  MS  define  block-upgrade  strategy 

IZIMS  monitor  logistics  data  /  spares,  consider  adding 
materiel  availability  (Am)  goal 
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Risk:  Cost  Increase 

Drivers: 

•  Resource  Management  Decision  (RMD)  802 
quantity  reduction  (C) 

•  Unknown  sustainment  strategy  (C) 

•  Business  Case  Analysis  (BCA)  timeline  impact 
to  POM-XY  (C) 

Recommendations: 

□  MS  budget  for  highest-cost  sustainment 
alternative,  expedite  BCA  analysis 


O  Initial  assessment:  Jan  20xx 
O  Current  assessment:  Feb  20xx 

0  Recommendation  shows  progress  and  /  or  completion 


Risk:  Initial  Operational  Capability  Schedule 

Drivers: 

•  Early  use  of  schedule  reserve  (S) 

•  Recent  training  delays  (S) 

•  Limited  Production  Qualification  Testing  (PQT) 
assets  (S) 

Recommendations: 

□  Program  office  perform  schedule  risk 
assessment 


Risk:  Program  Manning 

Drivers: 

•  MS  authorization  for  staffing  has  not  been 
approved  by  System  Center  (S,  P) 

•  NA-1  Aircraft  Product  Directorate  personnel 
turn-over  /  vacancies  (S) 

•  Competition  for  qualified  personnel  (S) 

Recommendations: 

□  MS  develop  high-priority  mitigation  plan  for 
manning  and  staffing 


2 

3  4 

Consequence 
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Example  -  Notional  Aircraft  (NA-1) 
4.1  Design  Consideration 


+  Positive 
O  Neutral 
-  Negative 
>"  Issue 
-  Risk 


□  Findings 

-  Current  Programmatic  Environment,  Safety,  and  Occupational  Health  (ESOH)  Evaluation  (PESHE)  document 
and  the  Federal  Aviation  Administration  (FAA)  Airworthiness  Certification  process  do  not  fully  address  the 
unique  safety  issues  of  Military  Operations  of  the  NA-1 

i"  The  PESHE  states  once  the  FAA  approves  the  NA-1 ,  the  aircraft  will  be  safe  for  humans,  but  this  does  not  fully  cover  ESOH 
risks.  Additionally,  an  FAA  airworthiness  certification  does  not  preclude  the  requirement  to  conduct  ESOH  analyses  necessary 
to  identify  hazards  and  associated  risks  using  MIL-STD-882D  methodology. 

-  Potential  for  NA-1  Program  Office  (PO)  to  improperly  identify  and  manage  ESOH  risks  with  potential  result  of  exposing 
personnel,  equipment,  and  the  environment  to  unknown  hazards. 

i"  The  PESHE  does  not  address  the  risk  of  continued  reliance  on  Halon  fire  suppression  systems. 

-  Potential  changes  in  FAA  certification  requirements  or  military  operational  risks  may  drive  changes  in  the  fire  suppression 
systems. 

□  Systemic  Analysis 

□  Root  Cause  Details:  Lack  of  substantiated  ESOH  hazard  /  risk  data  in  the  PESHE. 

□  Systemic  Root  Cause:  5.  Management 

□  Core  Root  Cause:  10.  Business  Practices 

□  First  Order  Impact 

□  Ineffective  ESOH  risk  management  resulting  in  the  potential  for  exposing  personnel,  equipment,  and  the 
environment  to  unidentified  hazards  with  potential  cost  and  /  or  schedule  implications. 

□  Recommendation 

□  Program  office  revise  the  PESHE  to  address  findings  above. 
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Common  PSR  ESOH  Observations  (Findings/Issues) 

►  ESOH  risk  data  and  technology  requirements  not  in  PESHE 

►  PESHE  does  not  describe  actual  ESOH  program 
implementation 

►  Program  Office  ‘System  Safety’  and  ‘ESOH’  efforts  not 
integrated 

►  Lack  of  emphasis  on  implementing  ESOH  mitigations 

►  Failure  to  address  USD  (AT&L)  hexavalent  chrome  policy 
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Value  of  ESOH  Participation  in  PSRs 

►  Increased  visibility  by  Program  Manager’s  has  lead  to 
increasing  ESOH  resources  (staffing  )  in  programs 

►  Improved  alignment  of  program  efforts  with  OSD  policy 

►  Improved  alignment  of  program  documentation  with  processes 
used  by  programs 

►  Program’s  reaching  out  for  additional  ESOH  support 
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A  Continuous  Improvement  Approach 
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Path  Forward 


►  Continue  to  provide  ESOH  Subject  Matter  Experts  to 
participate  on  PSRs 

►  Provide  support  to  ESOH  Practitioners  supporting  Programs 

►  Make  improvements  targeted  at  root  cause(s)  to  address 
repetitive  findings 

-  Policy  or  Guidance?  Share  Findings/Issues  with  DoD  Acquisition 
ESOH  IPT  members 

PESHE  content  improvements  and/or  PESHE  timing  ... 

Roles  and  responsibilities  ... 

-  Training  (i.e.,  CLE-009  update,  etc.) 
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Questions 

►  Government  Client 

-  David  Asiello 

-  ESOH  in  Acquisition  Lead 

-  ODUSD(l&E)/CMRM 

-  Phone:(703)604-1874 

-  Email:  David.Asiello@osd.mil 

►  Presenter 

-  William  A  Thacker  Jr 

-  Booz  Allen  Hamilton 

-  Phone:(703)412-7757 

-  E-Mail:  thacker  william@bah.com 


ODUSD(l&E)  /  CMRM 


Booz  Allen  Hamilton 


15 


BACKUP  SLIDES 
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Acquisition  ESOH  Policy  Vision 

►  As  part  of  sustaining  its  mission  DoD  is  committed  to  avoiding 

-  loss  of  life  or  serious  injury  to  personnel 

-  damage  to  facilities  or  equipment 

-  harm  to  the  environment  and  the  surrounding  community 

failure  with  adverse  impact  on  mission  capability,  mission  operability,  or  public 
opinion 

►  To  accomplish  this  in  systems  acquisition  we  must  use  the  System  Safety 
methodology  across  ESOH  disciplines  to  identify  hazards  and  mitigate 
risks  through  the  systems  engineering  process 

-  ESOH  refers  to  all  individual,  but  interrelated,  disciplines  that  encompass 
environment,  safety,  and  occupational  health 
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PSRs  Participation  Provides  Insight  to  Policy  Implementation 

►  Validate  program  compliance 

-  Determine  accuracy  of  PESHE  and  fill  in  unknowns 

►  Assess  effectiveness  of  Acquisition  ESOH  policy  and  re¬ 
enforce  reporting  of  High  and  Serious  category  ESOH  risks 
and  the  status  of  compliance  with  ESOH  technology 
requirements  at  program  reviews. 

-  DDR&E  prefers  this  approach 

►  Work  closely  with  program  teams  to  provide  ESOH  guidance 
and  direction 

-  Educates  the  work  force 

-  Establishes  an  “ESOH  network” 
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Policy  (DoDI  5000.02,  E12.6) 

►  Use  MIL-STD-882D,  DOD  Standard  Practice  for  System  Safety,  in  all  developmental 
and  sustaining  engineering  activities 

►  The  PM  must  report  the  status  of  all  High  and  Serious  ESOH  risks  and  applicable 
ESOH  Technology  Requirements  for  program  reviews  and  fielding  decisions 

►  Prior  to  exposing  people,  equipment,  or  the  environment  to  a  known  system-related 
ESOH  hazards, 

>  Risks  must  be  accepted  by  the  appropriate  authority 

>  User  concurrence  for  High  and  Serious  risks. 
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Taxonomy  of  Classifications 


+  Positive 
O  Neutral 
-  Negative 
■“  Issue 
-  Risk 


Note:  When  recording  multiple  negatives  in  a  PSR  report,  ensure  that  each  negative 
has  clear  linkage  with  its  risk  or  issue,  recommendation,  root  cause,  and  impact 
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Program  Support  Review  -  Definitions 


inding.  An  inquiry  by  the  program  support  review  team  into  a  DAPS  methodology  area,  sub-area, 
or  factor.  Findings  can  be  either  known  or  unknown  by  the  PMO  and  characterized  as. . . 

+  Positive.  Programmatic  or  technical  effort  that  is  above  normal  or  expected  effort,  and  which 
could  lead  to  a  strength  and/or  an  institutionalized  best  practice. 

O  Neutral.  Normal  programmatic  or  technical  effort.  May  be  a  candidate  for  process 
improvement. 

-  Negative.  Programmatic  or  technical  effort  that  is  lacking  positive  properties  or  may  introduce 
variation.  (Generally  stated  in  a  broad  manner,  similar  in  nature  to  the  statements  of  positive 
and  neutral  findings.)  Consequent  current  or  potential  future  problems  are  identified  as  issues 
or  risks,  with  at  least  one  issue  or  risk  being  identified  for  a  negative  finding.  Multiple  issues  or 
risks  may  be  associated  with  a  negative  finding. 

»“  Issue.  Current  problem  that  should  be  resourced  and  resolved. 

-  Risk.  A  future  uncertainty  relating  to  achieving  program  technical  performance  goals 
within  defined  cost  and  schedule  constraints.  Risks  are  associated  with  negative  findings 
or  may  be  associated  with  issues. 


+  Positive 
O  Neutral 
-  Negative 
>"  Issue 
-  Risk 
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Program  Support  Review  -  Definitions  (continued) 

Systemic  Analysis: 

■  Root  Cause.  Analysis  to  determine  the  underlying  reason  for  the  negative  finding  and  associated 
issue  or  risk.  The  root  cause  can  be  developed  using  5  “whys”  and  should  focus  on  addressing 
the  problem  and  not  the  symptom.  Three  tiers  of  root  cause  characterizations  are  required: 

■  Tier  1 :  Root  Cause 

■  Textual  description  aligns  with  DAPS;  documented  by  PSR  team 

■  Perceived  program  root  cause 

■  Tier  2:  Systemic  Root  Cause 

■  Short  descriptor  (from  pre-defined  list);  assigned  by  PSR  team 

■  Something  within  DoD  scope  to  solve.  Can  be  “Acquisition”  or  “acquisition” 

■  Tier  3:  Core  Root  Cause 

■  Short  descriptor  (from  pre-defined  list);  assigned  by  PSR  team 

■  Something  outside  the  Department.  Bigger  than  “Acquisition” 

■  First  Order  Impact.  The  programmatic  or  technical  effect  of  issue(s)  and/or  risk(s).  Viewed  from 
the  “first  order”  prior  to  performance,  cost,  or  schedule  changes. 

□  Recommendation.  Advice  or  additional  insight  on  how  to  resolve  negative  finding(s),  and  the 
associated  issue(s),  or  mitigate  risk(s). 
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+  Positive 
O  Neutral 
-  Negative 
>"  Issue 
-  Risk 
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Root  Cause  Analysis 


Systemic  Root  Causes 

Amplifying  Description 

1. 

Baseline  Management 

Baselines  not  stable  or  incomplete 

2. 

Communication 

Inadequate  external  information  flow  between  government  and  contractor,  or  internal  information  flow  at  the 

IPT  level 

3. 

Competing  priorities 

Need  vs.  Schedule  vs.  Cost  vs.  Performance  vs.  Technical  /  Integration  level  of  effort 

4. 

Contract  Structure  and  Execution 

Deliverables/Data  required  not  specified  /  Insufficient  Contract  Content  and  Structure 

5. 

Management 

Inadequate  Planning  /  Oversight  /  EVM  /  Cost  Accounting  /  Risk  mgmt  /  Supplier  mgmt  /  Accountability  / 
Definition  of  Enterprise  /  Tools 

6. 

Organization 

Inappropriate/Not  defined  /  Roles  and  responsibilities  /  Responsibility  w/o  Authority 

7. 

Acquisition  Practices 

Poor  Acquisition  practices  /  Fundamentally  flawed  application  of  practices 

8. 

Production 

Flow  /  Capacity  /  Process  Control  /  Process  Capability  /  Quality 

9. 

Program  Realism 

Unrealistic  expectations  /  Risk  acceptance/  Funding,  Budget,  and  Schedule  constraints  and  alignment  / 
Inadequate  Capital  investment  /  Poor  assumptions-  COTS,  TRL,  etc 

10. 

Requirements 

Ambiguity  /  Stability  /  JCIDS  /  No  SE  in  Requirements  process  /  CONOPS  incomplete 

11. 

Staff 

Qualifications  /  Skill  Availability  /  Experience  level  /  Continuity  /  Workload  /  Slots  /  Training 

12. 

Technical 

Poor  SE  /  Requirements  decomposition  /  V&V  /  Inadequate  system  Integration  /  Inadequate  Modeling  & 
Simulation  /  Logistics/Sustainment  late  to  need  in  SDD /  Poor  Life  Cycle  Planning 

13. 

Trade  Space  /  Constraints 

Excessive  Requirements  /  Insufficient  Resources  /  Insufficient  Stakeholder  involvement 

14. 

Other1 

If  “Other”  provide  description  of  desired  Systemic  Root  Cause  term 

15. 

Unknown2 

Unknown 
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Root  Cause  Analysis  (continued) 


Core  Root  Causes 

Amplifying  Description 

1 .  Acq  reform:  Loss  of  Gov’t  capital 
investment 

Inadequate  resources  (e.g.,  people,  facilities,  test  assets) 

2.  Acq  reform:  Loss  of  MS  A  requirement 

Programs  entering  late  and  with  less  maturity  into  acquisition  system 

3.  Acq  Reform:  Transferred  Authority 

Gov’t  transferred  too  much  authority  to  contractor  /  Gov’t  doesn't  provide 
enough  guidance  to  contractor 

4.  Budget  POM  process  (PBBE) 

Inadequate  funding  and/or  phasing  to  support  program 

5.  Culture 

Govt.  /  Industry  do  not  understand  each  other  /  have  different  motives 

6.  Enabling  Infrastructure 

Conditions  /  Constraints  affecting  programmatic  and  technical  effort 

7.  External  Influences 

Program  forced  to  make  decisions  about  cost,  schedule,  and  performance 
based  on  leadership/external  influences 

8.  JCIDS  process 

Capabilities  and/or  Requirements  not  tangible,  measurable,  or  reasonable 

9.  Human  Resource  Management 

Pool  of  clearable  skilled  people;  Gov’t.  /  Industry  lack  qualified,  cleared  staff  to 
support  effort  (e.g.  software  programmers);  Rotations  /  continuity  -  loss  of 
continuity  and  knowledge  base 

10.  Business  Practices 

Govt.  /  Industry  not  following  best  practices  /  Not  using  published  guides  to 
facilitate  program  and  technical  management 

11.  Other1 

Provide  description  of  desired  Core  Root  Cause  term 

12.  Unknown2 

Only  select  “Unknown”  if  a  root  cause  cannot  be  determined 

ODUSD(l&E)  /  CMRM 
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PSR  Participation 

►  Small  Diameter  Bomb  II 

►  H  C/M  C- 130 

►  C-27  Joint  Cargo  Aircraft  (JCA) 

►  Joint  Air  Ground  Missile  (JAGM) 

►  Joint  Air-to-Surface  Standoff  Missile 
Extended  Range  (JASSM-ER) 

►  F-35  Joint  Strike  Fighter  (JSF) 


ODUSD(l&E)  /  CMRM 


►  Global  Hawk 

►  MQ-9  Reaper 

►  Mobile  Landing  Platform  (MLP) 

►  Littoral  Combat  Ship  (LCS) 

►  B-2  EHF  Increment  1 

►  B61  Tail  Sub  Assembly 
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